Hilbert series, Howe duality, and branching rules.

Let lambda be a partition, with l parts, and let F(lambda) be the irreducible finite dimensional representation of GL(m) associated to lambda when l < or = m. The Littlewood Restriction Rule describes how F(lambda) decomposes when restricted to the orthogonal group O(m) or to the symplectic group Sp(m2) under the condition that l < or = m2. In this paper, this result is extended to all partitions lambda. Our method combines resolutions of unitary highest weight modules by generalized Verma modules with reciprocity laws from the theory of dual pairs in classical invariant theory. Corollaries include determination of the Gelfand-Kirillov dimension of any unitary highest weight representation occurring in a dual pair setting, and the determination of their Hilbert series (as a graded module for p(-)). Let L be a unitary highest weight representation of sp(n, R), so*(2n), or u(p, q). When the highest weight of L plus rho satisfies a partial dominance condition called quasi-dominance, we associate to L a reductive Lie algebra g(L) and a graded finite dimensional representation B(L) of g(L). The representation B(L) will have a Hilbert series P(q) that is a polynomial in q with positive integer coefficients. Let delta(L) = delta be the Gelfand-Kirillov dimension of L and set c(L) equal to the ratio of the dimensions of the zeroth levels in the gradings of L and B(L). Then the Hilbert series of L may be expressed in the form H(L)(q)=c(L) P(q)(1-q)(delta). In the easiest example of the correspondence L --> B(L), the two components of the Weil representation of the symplectic group correspond to the two spin representations of an orthogonal group.